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ABSTRACT 

This study examined how three dimensions of college 
students' experiences affect critical thinking skills: (1) curricular 
exposure, (2) formal classroom and instructional experiences, and (3) 
out-of-class experiences. The study used a i-year, longitudinal, 
panel study design using data collected, as part of a pilot study for 
a national, longitudinal investigation of factors influencing 
learning, cognitive development, and orientations toward learning in 
college. The survey randomly selected 600 students from a population 
of about A, 500 freshmen students at a large, urban, commuter, 
research university in Fall 1991. Of these, 327 (54.5V) participated 
and 210 (64.27.) of these also participated in the follow-up component 
in Spring 1992. The initial survey used the Collegiate Assessment of 
Academic Proficiency and also gathered demographic, family, and 
educational background information. The fcilow-up instruments 
included the College Student Experiences Questionnaire. Results 
indicated that college experiences gave a 6 to 17 percent increase in 
critical thinking skills, that both class-related and out-of-class 
experiences made positive, significant contributions to critical 
thinking scores, and that the number of hours students spent studying 
and the number of non-assigned books read were positively related to 
gains in critical thinking. (Contains 46 references.) (JB) 



Reproductions supplied by EDRS are 



be made K 



that can 



the best 

from the original document. 



OCR 

Deliverable 



INFLUENCES AFFECTING THE DEVELOPMENT 
OF STUDENTS' CRITICAL THINKING SKILLS 



by 

Patrick T. Terenzini 
Professor, Senior Scientist, and Associate Director 
National Center on Postsecondary Teaching, 
Learning, and Assessment 
The Pennsylvania State University 
403 South Allen St., Suite 104 
University Park, PA 16801-5202 
Telephone: (814) 865-5917 
E-Mail: PTT2@PSUVM.PSU.EDU 

Leonard Springer 
Research Assistant 
National Center on Postsecondary Teaching, 
Learning, and Assessment 
The Pennsylvania State University 

Ernest T. Pascarella 
Professor and Director 
National Study of Student Learning 
National Center on Postsecondary Teaching, 
Learning, and Assessment 
University of Illinois at Chicago 

Amaury Nora 
Associate Professor and Senior Research Associate 
National Study of Student Learning 
National Center on Postsecondary Teaching, 
Learning, and Assessment 
University of Illinois at Chicago 

Paper presented at the meeting of the Association for Institutional Research, New 
Orleans, LA, May 1993. 

The National Center on Postsecondary Teaching, Learning, and Assessment is funded by 
the U.S. Department of Education, Office of Educational Research and Improvement 
(OERI) under Grant No. R117G10037. The opinions herein do not necessarily reflect the 
position or policies of OERI, and no official endorsement should be inferred. 

PROJECT R117G10037 
CFDA 84.117G 



BEST COPY AVAILABLE 



INFLUENCES AFFECTING THE DEVELOPMENT 
OF STUDENTS' CRITICAL THINKING SKILLS 

Abstract 

This study estimates the relative and unique effects on changes in critical thinking 
of three dimensions of students' college experience: curricular exposure, formal 
classroom and instructional experiences, and out-of-class experiences. Students' 
classroom/ instructional and out-of-class experiences both make positive, statistically 
significant, and unique contributions to gains in critical thinking above and beyond 
students' precollege characteristics and level of critical thinking. Theoreticians have long 
speculated that students' academic and non-academic experiences jointly influence 
change, and this study supports that belief. The design and instruments in this study may 
be of interest to persons involved in assessment or the study of college impact. 



INFLUENCES AFFECTING THE DEVELOPMENT 
OF STUDENTS' CRITICAL THINKING SKILLS 



A relatively large body of evidence suggests that mach of the factual material 
students learn during college may have a relatively short shelf-iife after they leave 
(McLeish, 1968; Gustav, 1969; Blunt & Blizard, 1975; Brethower, 1977). Even if 
content material is not forgotten, however, it may soon become dated in many 
occupational and professional fields. Thus, higher education's claims to promoting 
student learning in the long run must be based on evidence that colleges and universities 
affect those learning skills that are likely to be enduring. It seems reasonable to suggest 
that critical thinking ability is one such enduring skill, that it is a central element in life- 
long learning, and that it is an appropriate (if not essential) skill for colleges and 
universities to develop among students. 

Pascarella and Terenzini (1991) note that "critical thinking" has been defined and 
measured in a number of ways "but typically involves the individual's ability to do some 
or all of the following: identify central issues and assumptions in an argument, recognize 
important relationships, make correct inferences from data, deduce conclusions from 
information or data provided, interpret whether conclusions are warranted on the basis of 
the data given, and evaluate evidence or authority" (p. 118). 

Although the definitions and measures of critical thinking vary across the available 
studies, the weight of evidence is generally consistent with respect to what does and does 
not influence gains in students' critical thinking abilities. These skills, for example, 
appear not to be differentially affected by a student's academic major. While Burns 
(1974), Bennett (1975-76), and King, Wood, and Mines (1990) all report differences 
across academic major fields, other studies (e.g., Pascarella, 1989; see also Pascarella & 
Terenzini, 1991) indicate these differences evaporate when precollege differences in 
students' academic aptitude or critical thinking skills are taken into account. 

Nor is there much evidence that critical thinking abilities are influenced by specific 
instructional variables (see McMillan, 1987 for a review of this literature). Pascarella 
and Terenzini (1991) suggest that one possible explanation for the absence of significant 
effects may be that a semester- or quarter-long course experience may be too brief to 
produce any measurable impact. 

While individual courses may not shape critical thinking, the number of courses 
taken in certain areas does appear to matter. For example, even after initial differences 
in aptitude or critical thinking ability are controlled, gains in critical thinking have been 
shown to be positively related to the number of science, music, literature, and art courses 
taken (Dressel & Mayhew, 1954; Pike, 1989; Pike & Phillippi, 1988; Pike & Banta, 
1989). Evidence from Winter, McClelland, and Stewart (1981) indicates gains may also 
be related to the interrelatedness of the courses taken, not just their number. In their 
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study, students who followed a couise of study that required the integration of ideas and 
coursies across disciplines showed greater gains on a measure of critical thinking than did 
students who took the regular courses in the same general areas but without the 
integrative emphasis. Forrest (1982) reports similar evidence. James Ratcliff and 
Elizabeth Jones have also found that certain sequences and combinations of courses across 
a rich array of disciplines, rather than any particular curricular structure, were related to 
gains it* analytical reasoning (Jones, 1992, p. 42). 

Other instruction-related factors also seem to make a difference. Although 
precollege critical thinking levels were uncontrolled, Smith (1977, 1981) found that 
(across both individuals and classes) three kinds of instructor-influenced classroom 
interactions were consistently and positively related to gains in critical think ing; the 
extent to which faculty members encouraged, praised, or used student ideas; the amount 
and cognitive level of student participation in class, and the amount of interaction among 
the students in a course. 

A growing body of evidence indicates a student's level of involvement both in and 
outside the classroom may have important effects on various forms of cognitive 
development, including critical thinking ability. For example, Gaff, Wilson, and their 
colleagues (Gaff, 1973; Wilson, Wood, & Gaff, 1974; Wilson, Gaff, Dienst, Wood, & 
Bavry, 1975) found level of student involvement in a variety of activities to be associated 
with several forms of cognitive development. A series of studies done at the State 
University of New York at Albany (e.g., Terenzini, Theophilides, & Lorang, 1984; 
Volkwein, Wright, & Agrotes, 1987; Terenzini & Wright, 1987) and a national study by 
Anaya (1989) ail contain evidence of the effects of students' out-of-class experiences on 
various forms of higher-order cognitive functioning. The most specific evidence relating 
to critical thinking comes from Pascarella (1989), who found that while each of nine 
measure of students academic and social experiences (e.g., living on-campus, 
interactions with faculty and peers, time spent studying) were not significantly related to 
gains in critical thinking (net of precollege levels), an aggregated measure of student 
social and academic involvement was a significant predictor, .suggesting that college's 
effects may be more cumulative and interrelated, rather than specific to any particular 
kind of experience. 

Similarly, evidence is mounting that students interactions with their peers and with 
faculty members (primarily, but not exclusively outside the classroom) are positively 
related to gains in general cognitive abilities. Evidence on the roles of these major 
agents of socialization are given in Ory and Braskamp (1988), Pace (1987, 1990), 
Terenzini and Wright (1987), Pascarella and Terenzini (1978), Terenzini and Pascarella 
(1980), Endo and Harpel (1982, 1983), Baxter Magolda (1987), and others. 

Virtually without exception, however, these and similar studies of the 
institutionally-controllable sources of influence on students' critical thinking skills have 



adopted a segmented approach in their conceptual and methodological designs. For 
example, as may be apparent in the foregoing literature review, the role of the curriculum 
is studied separately from the influences of methods of instruction, both of which are 
examined independently of classroom climate or instructor behaviors, and all these 
academic sources of influence on critical thinking are assessed as if students' out-of-class 
experiences were unrelated to gains in critical thinking. In short, while each of these 
areas of influence has a modest-to-large research base, these sources of influence on 
critical thinking have been studied as if they were independent of one another. 

This study estimates the relative importance of three sources of influence on 
students' critical thinking abilities during the first-year of college: students' course-taking 
activities, their formal instructional and classroom-related experiences, and their 
out-of-class experiences. The study's primary purpose is to estimate the extent to which 
critical thinking is shaped both independently and jointly by students' formal academic 
activities and out-of-class experiences. 



METHODS 

Conceptual Framework 

The basic conceptual model for this study (see Figure 1) is longitudinal and draws 
upon many of the elements of recent conceptualizations of college impact (e.g., Astin, 
1984; Pascarella, 1985; Tinto, 1975, 1987; Weidman, 1989). The model hypothesizes 
six sets of constructs defining a causal sequence that begins when students come to 
college with a wide array of educationally-relevant background characteristics. These 
precollege characteristics influence not only the outcomes of college directly, but also 
students' course-taking patterns, formal classroom experiences, and out-of-class 
experiences during college, which, in turn, also shape educational outcomes. The 
interplay between and among these sets of influences on learning takes place within a 
particular institutional context (e.g., organizational characteristics, structures, and 
policies). This study seeks to estimate the relative importance of students' curricular, 
classroom, and out-of-class experiences on learning-related attitudes and values after 
taking into account certain of the precollege characteristics of new students, including 
initial levels of interest in learning. (Because this is a single-institution study, the 
institutional context is constant for all students and, thus, cannot be a factor in differential 
change in students' critical thinking abilities.) 

Design. Sample, and Data Collection 

The study used a one-year, longitudinal, panel study design. Data were collected 
as part of a pilot study for a large, national, longitudinal investigation of the factors that 
influence learning, cognitive development, and orientations toward learning in college. 
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The population for the study was the approximately 4,500 students enrolled for six or 
more academic credit hours during their first semester (Fall, 1991) at a large, urban, 
Research I university in the midwest serving an undergraduate population composed' 
primarily of commuters. Students were recruited by mail and from the population of 
students attending precollege orientation. They were advised they would be participating 
in a national longitudinal study and would receive a st'pend for their participation. 
Students were also assured that the information they provided would be kept confidential 
and would never become part of their institutional records. The Fall, 1991 data 
collection required approximately four hours, and students were paid $35 for their 
participation. Students who participated in the Spring follow-up received a second 
stipend of $35 for their three and one-haif hours of testing. 

Of the approximately 1,150 new students who volunteered for the initial, precollege 
data collection, 600 were randomly selected to participate (the small initial sample size 
relative to the population reflects budgetary constraints on the pilot study). Of the 600 
students selected, 327 (54.5%) actually did so, with 210 (64.2%) of those also 
participating in the subsequent follow-up data collection in the Spring of 1992 (the end of 
the students' first year). These 210 students were reasonably representative of the 
institution's population of new students, although there was some potential bias. While 
the trends were not statistically significant, students in the sample, compared with the 
population from which they were drawn, had somewhat higher academic aptitudes and 
were somewhat more likely to be white than students of color. 

Variables 

Fall 1991 data were collected using two instruments. The first was Form 88B of 
the Collegiate Assessment of Academic Proficiency (CAAP), developed by the American 
College Testing Program to assess selected general skills typically developed by students 
in the first two years of college (ACT, 1990). The total CAAP consists of five, 40- 
minute, multiple- choice test modules (reading comprehension, mathematics, writing, 
science reasoning, and critical thinking). The second instrument was specifically 
designed for this study and gathered information on students' demographic, family, and 
educational backgrounds, as well as other precollege characteristics 

I ' 

The Spring, 1992 follow-up instruments included Form 88A of the CAAP, Pace's 
(1984) College Student Experiences Questionnaire (CSEQ) to measure students' first-year 
experiences in college, and a specially-designed follow-up survey form assessing aspects 
of students' first-year experiences not covered by the CSEQ. 

Following the conceptual framework for this study four sets of independent 
variables and one dependent variable and were developed. The first set of independent 
variables consisted of students' precollege characteristics, treated as control variables in 
this study. That set included parents' combined formal education and total family 
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income, and students' race/ethnicity, gender, degree aspirations, and precollege scores on 
the CAAP Critical Thinking Module. The operational forms of all these control variables 
are given in Table 1. Examination of the distributions of the categorical variables 
(race/ethnicity, gender, and highest degree planned) indicated that the limited skewness 
present was unlikely to bias regression parameter estimates. 



Insert Table 1 about here. 



As explained in greater detail below, the results of this study are based on two, 
"reduced model" multiple regressions containing only those independent variables which 
preliminary analyses indicated were related to the dependent measure. Thus, not all 
variables used in these preliminary analyses were retained for the final analyses. Table 1 
lists the variables (including item/scale content and metrics) in each of the three areas of 
institutional influence (the curriculum, class-related experiences, and out-of-class 
experiences) that were retained for the "reduced model" regressions on CAAP critical 
thinking scores. Table 2 lists the variables tested in the preliminary screening analyses 
but not retained for the final analyses. 



Insert Tables 2 about here. 



The dependent variable was critical thinking skills, as measured by the CAAP 
module of the same name. This module is a 32-item measure of students' abilities to 
clarify, analyze, evaluate, and extend arguments. The test consists of four passages that 
are representative of the kinds of issues commonly encountered in a postsecondary 
curriculum. A passage typically presents a series of subarguments that support a more 
general conclusion. Each passage presents one or more arguments and uses a variety of 
formats, including case studies, debates, dialogues, overlapping positions, statistical 
arguments, experimental results, or editorials. Each passage is accompanied by a set of 
multiple choice items. The KR-20 internal consistency reliability coefficients for the 
critical thinking test ranged from .81 to .82 (ACT, 1990, pp. 11-13, 33). Fk the follow- 
up data collection, Form 88A of the CAAP Critical Thinking Module was used (this form 
is psychometrically equivalent to that used in the Fall). 

Analytical Procedures 

The conceptual model underlying this study (see Figure 1) specifies causal relations 
among the three college experience variable sets. The analyses reported below were not 
intended to test those relations, but rather to estimate the unique and joint contributions of 
students' academic and out-of-class experiences to changes in students' orientations to 
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learning. Thus, hierarchical regression, rather than causal modeling, techniques were 
adopted. 

Data analysis proceeded in two stages. In order to avoid inflated estimates of the 
proportion of the variance explained due to the large number of independent variables 
relative to the sample size, the first stage consisted of a series of ordinary least-squares 
(OLS) regressions to identify those variables within each college experience set 
(curriculum, class-related, and out-of-class experiences) that were statistically significant 
predictors of each outcome measure after controlling for students' precollege 
characteristics, but not controlling for students' precollege critical thinking ability levels 
or other college experience variables. These variables were left uncontrolled to avoid 
masking (through collinearity among the predictor variables) the possible influence of 
college experiences that might be of theoretical or practical interest in their own right. 
For the same reason, any college experience variable related to critical thinking at p < 
.10 was retained. Thus, to avoid overlooking any variable with potential theoretical or 
practical importance, the variable selection process was an inherently lenient one. 

The second stage of analysis used ordinary least-squares multiple regression 
analysis to estimate the unique and joint proportion of the variance explained by each of 
the three (now reduced) college experience variable sets. To estimate the unique variance 
attributable to each category of variables, each of the three sets of college influence 
measures was entered into the regression after precollege characteristics and the other two 
college experience sets had been entered. The change in the value of the R 2 
accompanying the entry of the last set reflects the magnitude of that variable set's unique 
(or net) influence on critical thinking ability above and beyond that attributable to 
students' precollege characteristics and all other college experience variables. 

Estimates of the proportion of the total variance shared by the three college 
experience variable sets were derived arithmetically, not by the entry of a set of statistical 
(cross-product) interaction terms. Shared variance estimates were calculated by 
subtracting from the overall R 2 the sum of (a) the variance due to the precollege 
characteristics, and (b) the unique (R 2 -Change) variance associated with each of the three 
college experience variable sets. Such an analytical approach produces conservative 
estimates of the influence of each set of experience variables in that any variance these 
experience variables share with students' background characteristics are attributed to the 
precollege characteristics set. 

Students' precollege critical thinking ability could be expected to be the single-most 
pow. .till predictor of critical thinking skill at the end of the first year. Under such 
conditions, the probability was high that the influence of other predictor variables of 
theoretical or practical interest might be masked due to collinearity among the 
independent variables. Consequently, two "reduced model" regressions (i.e., containing 
only those variables identified in the first stage analyses) were run. In the first (the 
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"Out" model), precollege critical thinking ability was left uncontrolled. In the second 
(the "In" model), precollege critical thinking ability was included in the set of precollege 
characteristics (i.e., controlled). The two reduced models produce something 
approximating upper- and lower-bound estimates of the influence of each variable set. 
Inclusion of students' precollege level of critical thinking skills (the "In" model) probably 
underestimates college's influence, while exclusion of precollege critical thinking ability 
(the "Out" model) probably overestimates college's effects. 



RESULTS 

Table 3 reports the results of the two reduced-model regressions partitioning the 
total variance explained into that attributable to student's precollege characteristics and to 
each of the three college experiences variable sets. As can be seen there, the model with 
students' precollege critical thinking ability uncontrolled explained 30.1 percent of the 
total variance, while the model with that variable included explained 52 percent of the 
total variance. Both amounts are statistically significant at p < .001. 

More important, students' course-related and out-of-class experiences both made 
unique and statistically significant (if modest) contributions to the variance explained 
above and beyond that attributable to students' precollege characteristics or other college 
experiences and regardless of whether initial critical thinking ability was taken into 
account. Above and beyond students' precollege characteristics, their college experiences 
explained an additional 7% to 17% of the variance in first-year gains in critical thinking 
(depending on whether precollege critical thinking skills were controlled). Interestingly, 
even with precollege critical thinking controlled, students' out-of-class experiences 
contributed as much to gains in critical thinking as did students' class-related experiences 
(2.9% and 2.5%, respectively). The number of courses students took in different areas 
was related to critical thinking at the end of the first year, but that effect disappeared 
when precollege critical thinking ability was controlled (the "In" model). As one would 
expect, the variance explained by students' class-related and out-of-class experiences was 
reduced (by about half) when precollege critical thinking ability was controlled. 

Terenzini, Springer, Pascarella, and Nora (in press) found that these same three 
variable sets made modest shared contributions to the variance explained in two measures 
of students' orientations toward learning over and above that attributable to precollege 
characteristics or any single set of college experiences (range = 1.8% to 11.7%). The 
shared variance terms in this study, however, are virtually nil. 

Table 4 reports the standardized regression coefficients (beta weights) reflecting the 
relative contributions of each component variable to the explanation of variance in year- 
end critical thinking ability. With or without precollege critical thinking skill controlled, 
one precollege characteristic, one class-related variable, and two out-of-class experience 
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variables had significant and unique effects on year-end critical thinking. Parents' 
education, the number of hours students spent studying and the number of non-assigned 
books they read during the year were all significantly and positively related to first-year 
gains in critical thinking. Students* relationships with their peers, however, was 
negatively related to gains in critical thinking. This variable is a single-item scale on 
which students are asked to rate their "relationship with other students, student groups, 
and activities" on a 7-point scale where 1 = "competitive, uninvolvti" and 7 = 
"friendly, supportive, sense of belonging." 

Several other college experience variables were significantly and uniquely related to 
year-end critical thinking ability in the "Out" model. Students' reports of the 
effectiveness of their instruction in social science courses and the number of science 
courses taken were both positively related to year-end critical thinking levels. The 
. number of courses taken in mathematics and students' library experiences, however, were 
both significantly and negatively related to year-end critical thinking. None of these 
effects remained, however, when precollege critical thinking ability was controlled. 

Limitations 

This study is limited in several respects. First, it is based on data from a relatively 
small sample of students at a single institution who are probably not representative of any 
national population. Similarly, only a small number of these students lived in university- 
controlled housing; thus, the nature and impact of their college experiences may not be 
representative of those of students at residential institutions. These students, however, 
may well be representative of first-year students at similar commuter institutions. 
Second, the study examines change over only one year. It seems quite possible (even 
probable) that greater, cumulative changes in critical thinking skills may occur over the 
full course of students' college careers. This study, however, cannot address questions 
* relating to either the magnitude of change over a longer period or whether the same 
college influences may also be salient in subsequent college years. Third, "critical 
thinking" is a complex construct, and a variety of definitions and ways of measuring it 
have been advanced (see, for example, McMillan, 1987). There is also some dispute 
over whether critical thiucing is a general cognitive ability (as it is assumed to be in this 
study), or a skill that varies in its character across disciplines. Fourth, the 'measures of 
students' course-taking experiences (the number of courses taken in each of six general 
disciplinary categories) probably does not adequately reflect the effects of those courses 
(or the patterns among them) on changes in students' critical thinking skills. Finally, the 
measures of instructor effectiveness is based on students' perceptions of instructional 
competence and not on some more objective measure. 
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DISCUSSION AND CONCLUSIONS 

Overall, the two, "reduced model" regressions explained 30% and 52% of the 
variance in critical thinking abilities, depending on whether precollege critical thinking 
skill was included in the model. Above and beyond students' precollege characteristics, 
their college experiences explained an additional 6% or 17% of the variance in first-year 
gains in critical minking (again, depending on whether precollege scores were included in 
the model; both increments are statistically significant). It is important to note that 
students' class-related and out-of-class experiences both made positive, statistically 
significant, and unique contributions to freshman year-end critical thinking scores, even 
after controlling students' precollege characteristics (including initial critical thinking 
scores) and other collegiate experiences. Moreover, these unique contributions to gains 
in critical thinking were equal in magnitude. Net of all other variables, the courses 
students took during their first year were not related to gains in critical thinking abilities. 
As noted above, however, this finding may be artifactual, due more to the relatively 
imprecise measurement of auricular effects (i.e., number of courses taken was used 
instead of, say, coursework patterns). 

As might be expected, the number of hours students spent studying and the number 
of non-assigned books read during the year were positively related to gains in critical 
thinking. In contrast, gains were negatively related to students' perceptions of the quality 
of their relationships with student peers. Students who characterized their relationships 
with other students as "competitive, uninvolved, . . . alienai(ed)" were more likely to 
show gains in critical thinking than were students who portrayed their peer relations as 
"friendly, supportive, (or) a sense of belonging." The data in this study do not permit 
confident explanation of this relation, but one might speculate that a sense of participation 
and belonging in a "friendly, supportive" peer environment may require a partial 
suspension (or at least not the encouragement) of one's critical (i.e., analytic) thinking 
skills. Supportive peer environments, one might suggest, are more likely to promote the 
development of tolerance, compromise, consensus-building, and an emphasis on shared 
similarities rather than differences. Initial differences in critical thinking abilities cannot 
be offered as an explanation inasmuch as the relation persisted even when those 
differences were statistically controlled. ^ 

One must, of course, be mindful of the constraints on the generalizability of these 
finding. Nonetheless, the fact that students' class-related and out-of-class experiences 
both had effects on gains in critical thinking that were unique, statistically significant, and 
comparable in magnitude - even with precollege ability levels controlled - is both 
theoretically and practically suggestive. The finding is conceptually important, we 
believe, inasmuch as theoreticians (e.g., Smford, 1964; Chickering, 1969; Heath, 1968, 
1978) have long suggested that students learn and change in holistic ways and as a 
consequence of multiple college influences. Pascarella and Terenzini (1991) found clear 
evidence that various dimensions of students' collegiate experiences influenced how 
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students learn and change, but they also noted that, almost invariably, these sources of 
influence had been studied independently of one another. They found no empirical 
evidence to support beliefs about the holistic effects of the college experience on student 
learning. The evidence from this study. quite clearly indicates that changes in students' 
critical thinking abilities are shaped, independently, by what happens to them both in and 
out of the classroom. The findings of this study are consistent with those of Terenzini, 
Springer, Pascarella, and Nora (in press), who found similar unique class-related and out- 
of-class experience effects on students' attitudes toward learning. The two studies are not 
consistent, however, in that the earlier research also found a joint effect attributable 
simultaneously to students' course-taking, class-related, and out-of-class experiences 
above and beyond those attributable uniquely to any of those variable sets. No such joint 
effect was observed in this study. 

These findings suggest that future research on college impacts will have to be more 
comprehensive in both conception and design. Failure to take into account the multiple 
sources of influence that span the entire college experience is likely to result in 
incomplete representations of the college experience, misunderstanding of the web-like 
character of college's effects on scadents, and the underestimation of the magnitudes of 
those effects. 

From a practical point of view, these findings have important implications for how 
colleges and universities currently organize themselves and for how they structure 
students' learning experiences (e.g., the separation of academic and student affair 
divisions). Institutional and academic program planning processes, these findings 
suggest, are more likely to be successful and effective if they take into account the 
potential for simultaneous contributions of students' class-related and out-of-class 
experiences on student learning. Gains in critical thinking appear to be a consequence of 
a variety of student experiences, not just those that are part of the formal instructional 
program. Ways must be found to overcome the artificial, organizational bifurcation of 
our educational delivery systems. Academic and student affairs units have common 
goals, and the evidence of this study suggests that students are more likely to benefit 
educationally if these units work together, rather than separately, in pursuit of those 
common goals. 



!3 



- 11 - 



REFERENCES 



American College Testing Program. (1989). Report on the technical characte ri stics of 
CAAP: Pilot vear 1. 1988-89 . Iowa City, IA: Author. 

Anaya, G. (1989). Students' perceived cognitive change. Paper presented at the 
meeting of the American Educational Research Association, San Francisco. 

Astin, A. (1984). Student involvement: A developmental theory for higher education. 
Journal of College Student Personnel . 25, 297-308. 

Baxter MagoJda, M. (1987). The affective dimension of learning: Faculty-student 
relationships that enhance intellectual development. College Student Journal . 21 , 
46-58. 

Bennett, M. (1975-1976). A study of the relationship between curricula taken and the 
critical thinking abilities of high school seniors and University of Illinois freshmen, 
sophomores, and seniors majoring in elementary education. Dissertation Abstracts , 
3£, 5799A. 

Blunt, M., & Blizard, P. (1975). Recall and retrieval of anatomical knowledge. British 
Journal of Medical Education , 9, 255-263. 

Brethower, D. (1977). Research in learning behavior: Some implications for college 
teaching. In S. Scholl and S. Inglis (Eds.), Teaching in higher education . 
Columbus: Ohio Board of Regents. 

Burns, R. (1974). The testing of a model of critical thinking ontogeny among Central 
State College undergraduates. Dissertation Abstr ac ts International , 54, 5467A. 

Chickering, A. (1969). Education and identity . San Francisco: Jossey-Bass. 

Dressel, P., & Mayhew, L. (1954). General education: Explorations in evaluation . 
Westport, CT: Greenwood. 

Endo, J., & Harpel, R. (1982). The effect of student-faculty interaction on students' . 
educational outcomes. Research in Higher Education , 16, 115-138. 

Endo, J., & Harpel, R. (1983). Student-faculty interaction and its effect on freshman 
year outcome* at a major state university. Paper presented at the meeting of the 
Association for Institutional Research, Toronto. 



ERIC 



14 



- 12 - 



Forrest, A. (1982). Increasing student competence and persistence: The best case for 
general education . Iowa City, IA: American College Testing Program. 

Gaff, J. (1973). Making a difference: The impacts of faculty. Journal of Higher 
' Education . 44, 605-622. 

Gustav, A. (1969). Retention of course material over varying intervals of time. 
Psychological Reports . 25, 727-730. 

Heath, D. (1968). Growing up in college . San Francisco: Jossey-Bass. 

Heath, D. (1978). A model of becoming a liberally educated and mature student. In C. 
Parker (Ed.), Encouraging development in college students . Minneapolis: 
University of Minnesota Press. 

Jones, E. (1992). Is a core curriculum best for everybody? In J. Ratcliff (Ed.), 
Assessment and curriculum reform (pp. 37-46). New Directions for Higher 
Education, No. 80. San Francisco: Jossey-Bass. 

King, P., Wood, P., & Mines, R. (1990). Critical thinking among college and graduate 
students. Review of Higher Education . 13, 167-186. 

McLeish, J. (1968). The lecture method (Cambridge Monograph on Teaching Methods, 
No. 1). Cambridge, England: Cambridge Institute of Education. 

McMillan, J. (1987). Enhancing college students' critical thinking: A review of 
studies. Research in Higher Education . 26, 3-29. 

Ory, J., & Braskamp, L. (1988). Involvement and growth of students in three academic 
programs. Research in Higher Education . 28. 116-129. 

Pace, C. (1984). Measuring the quality of college student experiences . Los Angeles: 
University of California, Higher Education Research Institute. ^ 

Pace, C. (1987). Good things go together . Los Angeles: University of California at 
Los Angeles, Center for the Study of Evaluation. 

Pace, C. (1990). The undergraduates: A report of their activities and progress in 
' colleges in the 1980s . Los Angeles: University of California at Los Angeles, 
Center for the Study of Evaluation. 



15 



- 13 - 



Pascarella, E. (1985). College environmental influences on learning and cognitive 
development: A critical review and synthesis. In J. Smart (Ed.), Higher 
education: Handbook of theory and research (Vol. 1). New York: Agathon. 

Pascarella, E. (1989). The development of critical tWnking: Does college make a 
difference? Journal of College Student Development . 30, 19-26. 

Pascarella, E., & Terenzini, P. (1978). Student-faculty informal relationships and 
freshman year educational outcomes. Journal of Educational Research . 71, 183- 
189. 

Pascarella, E., & Terenzini, P. (1991). How college affects studen ts: Findings and 
insi ghts from twenty years of research . San Francisco: Jossey-Bass. 

Pike, G. (1989). Background, college experiences, and the ACT-COMP exam: Using 
construct validity to evaluate assessment instruments. Review of Higher Education. 
13, 91-117. 

Pike, G., & Banta, T. (1989). Using construct validity to evaluate assessment 
' instruments: A comparison of the ACT-COMP exam and the ETS Academic 
Profile . Paper presented at the meeting of the American Educational Research 
Association, San Francisco. 

Pike, G., & Phillippi, R. (1988). Relationships between self-reported coursework and 
' performance on the ACT-COMP exam: An analysis of the g eneral izahilitv of the 
differential coursework methodology . Paper presented at the meeting of the 
Association for the Study of Higher Education, St. Louis. 

Sanford, N. (1962). Developmental status of the entering freshman. In N. Sanford 
(Ed.), The American college: A psychological and social inte rpretation of the 
hi gher learning . New York: Wiley. 

Smith, D. (1977). College classroom interactions and critical thinking. Journal of 
Educational Psychology . 62, 180-190. ' 

Smith, D. (1981). Instruction and outcomes in an undergraduate setting. Paper 

presented at the meeting of the American Educational Research Association, Los 
Angeles. 

Terenzini, P., & Pascarella, E. (1980). Student/faculty relationships and freshman year 
educational outcomes: A further investigation. Journal of C ollege Student 
Personnel . 21, 521-528. 



1.6 



- 14- 



Terenzini, P., Springer, L., Pascarella, E., & Nora, A. (in press). Academic *nd out- 
of-class influences affecting the development of students' intellectual orientations. 
Review of Higher Education . 

Terenzini, P., Theophilides, C., & Lorang, W. (1984). Influences on students' 

perceptions of their academic skill development during college. Journal of Higher 
Education , 55, 621-636. 

Terenzini, P., & Wright, T. (1987). Influences on students' academic growth during 
four years of college. Research in Higher Education , 26, 161-179. 

Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent 
research. Review of Educational Research , 45, 89-125. 

Tinto, V. (1987). Leaving College: Re - hinking the causes and cures of student 
attrition . Chicago: University of Chicago Press. 

Volkwein, J., Wright, T., & Agrotes, M. (1987). The impact of college experiences on 
the intellectual growth of transfer students. Paper presented to the meeting of the 
North East Association for Institutional Research, Rochester, NY. 

Weidman, J. (1989). Undergraduate socialization: A conceptual approach. In J. Smart 
(Ed.), Higher education: Handbook of theory and research (Vol. 5). New York: 
Agathon. 

Wilson, R., Gaff, J., Dienst, E., Wood, L., & Bavry, J. (1975). College professors 
and their impact on students . New York: Wiley-Interscience. 

Wilson, R., Wood, L., & Gaff, J. (1974). Social-psychological accessibility- and 

faculty-student interaction beyond the classroom. Sociology of Education , 47, 74- 
92. 

Winter, D., McClelland, D., & Steward, A. (1981). A new case for the liberal arts: 
Assessing institutional goals and student development . San Francisco: Jossey- 
Bass. 



Table 1 

independent Variables in Critical Thinking Regression Mode? 



Category /Variable 



Precollege 

Parents ' Education : Sum of mother's and father's education 

on a 9-point scale , where 1 = grammar school or less and 
9 = professional degree. 

Total Family income : 14-point scale , where 1 = less than 
$6,000 and 14 = $150 , 000 or more. 

Race : 0 = nonwhite, 1 = white. 

Sex : 0 = female, 1 = male. 

Highest Degree Planned : 5-point scale, where 1 = none and 5 
= doctoral degree (Ph.D., Ed.D. , M.D., D.O., D.D.S., or 
D.V.M.). 

Out-of -Class Experiences 

Relationship with Students : CSEQ single-item rating of 

"Relationship with other students, student groups, and 
activities" on a 7- point scale, where 1 = "competitive, 
uninvolved, sense of alienation" and 7 = "friendly, 
supportive, sense of belonging." 

No, of Non -Assigned Books Read : CSEQ single-item rating on a 
5-point scale of the number of non-assigned books read 
during the current school year, where 1 = none and 5 = 
more than 20. 

Class-Related Experiences 

Hrs./Wk. Studying : Single-item rating on a 7-point scale 
where 1=0 hrs./wk. and 7 = more than 20 hrs./wk. 

instructor Effectiveness in Social Science : Single-item 
rating on a 5-point scale reflecting instructor's 
overall teaching effectiveness in the first course in 
social science taken at this college, where 1 = very 
poor and 5 = excellent. 
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Table 2 

« 

Independent Variables propped from Redu ced Model 



Category /Variable 



Out-of -Class Experiences 

Hours worked on-campus 

Hours worked off-campus 

Hours socializing with students 

Hours talking with teachers outside of class 

CSEQ scales: 

Athletic and Recreation Facilities 
Art, Music, Theater 
* Campus Residence 
Student Union 
Personal Experiences 
Clubs and Organizations 
Student Acquaintances 
Topics of Conversation 
Class-Related Experiences 

Number of textbooks or assigned books read 

Number of essay exams taken ! 

Number of term papers or other written reports 

Relationship with faculty 

Instructor effectiveness in science 

Instructor effectiveness in arts and humanities 

Instructor effectiveness in mathematics 
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Table 1 (Continued) 

Independent Variables in Critical Thinking Regressio n Model 



Category /Variable 



Class-Related Experiences (Continued) 



Library Experiences : 10-item CSEQ "Library Experiences" 
scale reflecting students' experiences in libraries 
(e.g. f "Used the library as a quiet place to read or 
' study materials you brought with you" and "Used the card 
catalogue or computer to find what materials there were 
on some topic"). Scored on a 4-point scale, where 1 « 
never and 4 = very often. Alpha « .81. 



Courses 



Mathematics : Number of college courses taken in geometry, 
calculus, matrix algebra, statistics, accounting, 
business math, pre-algebra, or algebra. 

Sciences: Number of college courses taken in astronomy, 
biology, botany, chemistry, engineering, geology, 
microbiology, physics, zoology, or other sciences. 

Arts and Humanities : Number of college courses taken in 
drawing, art history, applied art, studio art, dance, 
theater, music appreciation, music performance, English 
literature, foreign language, humanities, philosophy, 
linguistics, classics, religious studies, or 
communications • 
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Table 3 

Partitioning of variance Results for Reduced-Model Regression on 
Critical Thinking 



Variable 


Out« 


Inb 


Variance {iue to Precollege Characteristics 


.131*** 


.456*** 


Uniauec Variance due to: 






j_ flaacs 1?Tnori pnrP 5 ? 


.069*** 


.029** 


Class-Related Experiences 


.050** 


.025* 


Courses Taken 


.045** 


.014 


Total Shared Variance** 


.006 


.000 


Total Variance Explained 


.301*** 


.520*** 



•Precollege critical thinking score excluded from model, 
bprecollege score on dependent variable included in model. 
^Controlling for precollege characteristics and other college 

experience variable sets. 
^Shared among the three college experience variable sets; 

statistical significance cannot be tested. 

*p < .05.- **p < .01. ***p < .001. 
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Table 2 (Continued) 

Independent Variables Dropped from Reduc ed Model 



Category /Variable 

Class-Related Experiences (Continued) 

CSEQ scales: 

Experiences in Writing 
Experiences with Faculty 
Course Learning 
Science 

Courses 

Number of college courses taken ins 
composition or writing 
social science 
technical or preprofessional 



Table 4 



Beta Weights at Final Step for Reduced-Model Reg ress ion on Year- 
End Critical Thinking Ability 



Variable 


Out* 


Inb 


Precollege 






Parents' Education (Sum) 


.19** 


.14* 


Total Family Income 


.06 


-.00 


Race 


.12 


.00 


Sex 


.10 


.03 


Highest Degree Planned 


-.11 


-.09 


Initial Critical Thinking Ability 




.54*** 


Out-of -Class Experiences 






Relationship with Students 


-.17** 


-.14** 


No. Non-Assigned Books Read 


.21*** 


.11* 


Class-Related Experiences 






Hrs./Wk. Studying 


.17** 


.14** 


Instructor Effectiveness 
in Soc. Science 


.13* 


.06 
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Courses 






No. in Mathematics 


-.17* 

I 


-.11 


No. in Sciences 


.18* 


.10 


No. in Arts and Humanities 


.12 


.06 



aprecollege critical thinking score excluded from model. 
^Precollege critical thinking score included in model 



*r> < .05. **p < .01. ***p < .001. 
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